Outlier Detection on High Dimensional Data Using RNN

Background: Outlier detection is an important factor in data mining since it is used in various real time applications. Outlier is an extreme points that are not related to any of the class. Dealing with dimensions is the great challenge, due to “curse of dimensionality”, for effective outlier detection. In a high dimensional data space, it is difficult to detect most related points and most unrelated points. Outlier is the most unrelated points and in the high dimensional data all data points seemed to be a good outlier, which is a great challenge to identify. Objective: Reverse nearest neighbor technique is used to determine the occurrence of the points in K-nearest neighbor list. The most frequently occurring points are referred as hub points and rarely occurring points are referred as anti-hub points. Results: Unsupervised learning helps to find the clear outliers, throughout this paper we deal with both synthetic data and real data, to detect the clear outliers. Based on analysis, the proposed work shows that RNN-distance based similarity provides higher percentile score to detect outliers when compared to basic KNN approach and ABOD method. In addition ID3 algorithm is used to improve the detection of outlier points. This ID3 algorithm works good for all real datasets as well as synthetic datasets.